Perturbed Markov chains
نویسندگان
چکیده
We obtain results on the sensitivity of the invariant measure and other statistical quantities of a Markov chain with respect to perturbations of the transition matrix. We use graph-theoretic techniques, in contrast with the matrix analysis techniques previously used. keywords: Perturbed Markov chains, conductance. Primary subject classification: 60J10 Secondary subject classification: 60F10 ∗MEDS Department, Kellogg School of Management, Northwestern University, and the School of Mathematical Sciences, Tel Aviv University, Tel Aviv 69978, Israel. e-mail: [email protected] †Département Finance et Economie, HEC, 1, rue de la Libération, 78 351 Jouy-enJosas, France. e-mail: [email protected] ‡We acknowledge the financial support of the Arc-en-Ciel/Keshet program for 2001/2002. The research of the second author was supported by the Israel Science Foundation (grant No. 03620191).
منابع مشابه
Singularly Perturbed Finite Markov Chains with General Ergodic Structure
We analyse singularly perturbed Markov chains. Most previous research has been done under the assumption that the perturbed Markov chain is either ergodic or unichain. In this paper we do not impose any restrictions on the ergodic structure of the perturbed chain. The present approach is based on the inversion of analytic matrix-valued functions.
متن کاملErgodic Control of a Singularly Perturbed Markov Process in Discrete Time with General State and Compact Action Spaces
Ergodic control of singularly perturbed Markov chains with general state and compact action spaces is considered. A new method is given for characterization of the limit of invariant measures, for perturbed chains, when the perturbation parameter goes to zero. It is also demonstrated that the limit control principle is satisfied under natural ergodicity assumptions about controlled Markov chain...
متن کاملProof of the Hamiltonicity-trace Conjecture for Singularly Perturbed Markov Chains
We prove the conjecture formulated in Litvak and Ejov (2009), namely, that the trace of the fundamental matrix of a singularly perturbed Markov chain that corresponds to a stochastic policy feasible for a given graph is minimised at policies corresponding to Hamiltonian cycles.
متن کاملStrong Stability and Perturbation Bounds for Discrete Markov Chains
We consider the stability problem of discrete Markov chains when their transition matrices are perturbed. For finite irreducible Markov chains many perturbation bounds for the stationary vector are available. In this paper, we identify a condition under which these bounds are carried over to discrete irreducible Markov chains which are not finite.
متن کاملStationary Distributions and Mean First Passage Times of Perturbed Markov Chains
Stationary distributions of perturbed finite irreducible discrete time Markov chains are intimately connected with the behaviour of associated mean first passage times. This interconnection is explored through the use of generalized matrix inverses. Some interesting qualitative results regarding the nature of the relative and absolute changes to the stationary probabilities are obtained togethe...
متن کاملControl of Singularly Perturbed Markov Chains: a Numerical Study
This work is devoted to numerical studies of nearly optimal controls of systems driven by singularly perturbed Markov chains. Our approach is based on the ideas of hierarchical controls applicable to many large-scale systems. A discrete-time linear quadratic control problem is examined. Its corresponding limit system is derived. The associated asymptotic properties and near optimality are demon...
متن کامل